Spectral Line Broadening with Transform Domain Additive Synthesis

After a survey of inverse transform methods for the efficient synthesis of narrow-band and broad-band signals, a novel spectral line broadening technique is introduced for synthesis of pitch modulated noise signals. This new transform-domain approach is compared to the time-domain oscillator method with respect to their relative efficiency on modern processors Introduction: Noise in Musical Instrument Sounds The term “noise” is used to describe the perception of a multitude of features of sounds from musical instruments, for example: • Dense modes, e.g., cymbals • Additive “noise” from turbulence in blown instruments such as the flute or consonants in the voice. • Impulses from short-term interactions such as hammer strikes, string plucks, key and tone hole closure and openings. • Bandwidth broadening from non-linear mechanisms such as piano dampers, harpsichord quills, tampoura and the sarod jawari bridge. • Correlated or convolutional noise in blown instruments where a reed (or vocal fold) gates or modulates a turbulent noise source. This is also observed in bowed instruments and flue pipes. • Impulse bursts as found in maracas, cabasa, and washboard. • Non-linear oscillator noise generated within the oscillator itself (chaos). The Sum of Sinusoid+Residual models of McAulay/Quatieri, Serra/Smith, Depalle/Rodet, et al., have proved useful for modeling and coding short musical tones. The assumption of these models is that the residual is colored independently of sinusoidal parameter estimates. This assumption is invalid for most musical instruments so inadequate fusion of re-synthesized noise and sinusoidal components is often observed. This is especially troublesome when transformations are applied such as time scaling and pitch shifting (Laroche, 1993, Laroche and Dolson, 1997, Laroche, et al., 1993). The problem is that all forced oscillators (bowed strings, voice, reeds, trumpets, flue pipes, etc.) generate nearly-periodically modulated noise, not additive noise. A combination of a better understanding of the physics of these oscillatory mechanisms (Rodet, 1993, Rodet, 1995) and new methods in higher order statistics (Brillinger and Irizarry, 1998, Dubnov and Rodet, 1997), wavelets (Goodwin and Vetterli, 1996) and time series (Irizarry, 1998) are leading to better tools for multilevel decomposition of sounds into transient events, pitched and unpitched oscillations, convolutional noise and colored noise. These new models require efficient, real-time noise synthesis algorithms. This paper contributes an efficient implementation of one such algorithm for noise synthesis: spectral line broadening. Line Broadening Modulating the phase of a sinusoidal carrier with a random signal results in a narrow band noise source. This spectral broadening process has been used for decades in spread-spectrum radio frequency (RF) communications systems where it is usually implemented directly in the time domain. Musical applications of line broadening were explored by Risset and Wessel in the 1970’s (Risset and Wessel, 1982). With appropriate parameters for the noise amplitudes, sounds synthesized using spectral line broadening processes are perceived as similar to the noise found in voice and musical instruments such as flutes and flue pipes. Since the two noise generating mechanisms are quite different, it is interesting to consider what features the mechanisms have in common that may explain a similar percept. In the voice and aformentioned wind instruments, the noise process is the result of turbulence, the amplitude of which is dependent on air velocity, which is modulated by the nearly periodic primary oscillator. The fundamental frequency and partial amplitudes are not greatly influenced by the turbulence. This independence is a feature of the spectral line broadening process because of the use of a zero mean random phase modulation. In physical systems the amplitude of the primary oscillator and turbulent noise are both proportional to driving energy. The amplitude parameter of the line broadening spectral synthesis process conveniently adjusts the amplitude of both elements. This parameterization is a convenient starting point for more sophisticated musical instrument models that dose noise and partial energy according to frequency and driving force. A final important connection between sounds created by spectral line broadening and modulated noise is that both are perceived as originating from a single source. In contrast to additive noise models, the integrity of spectral line broadened sources survives musically useful transformations such as transposition, time dilation and compression. Implementation Implementing spectral line broadening efficiently with oscillator methods on modern, generalpurpose microprocessors is surprisingly challenging. The first problem is that most pseudorandom sequence generators employ integer arithmetic operations, which are slower than floating point multiply/add operations on most processors. The second problem is that the noise signals have to be scaled to dose the line broadening before being added to the current phase (or frequency) of the oscillator. The scaling is fastest in floating point arithmetic, but on common processors, such as the PowerPC, conversion of the final phase back to an integer (for the sinusoidal table lookup) is prohibitively expensive. No provision for spectral line broadening has been made to date in custom VLSI real-time systems for additive synthesis of music (De Bernardinis, et al., 1997, Honghton, et al., 1995, Phillips, et al., 1997). One reason for this is that the interface between the musical control software and the synthesis circuits is the primary performance bottleneck and increasing the number of parameters to send across this interface worsens the problem. Transform-domain synthesis methods are an effective alternative to time-domain oscillators. Because they exhibit good temporal and spatial locality, implementations of transform-domain algorithms can exploit the register, cache and main memory hierarchy of modern computers. Communication bottleneck can be minimized by computing the control and synthesis functions in a single address space and by computing control functions at a frame rate, typically around 1/100 of the output sample rate. After the following survey of additive synthesis techniques, we present a new algorithm for spectral line broadening using transform domain techniques. Survey In the late 1970’s, the availability of single chip digital multipliers stimulated the construction of digital signal processors for musical applications (Allen, 1985). Although these machines were capable of accurately synthesizing hundreds of sinusoids (DiGiugno, 1976), their prohibitive cost and limited programming tools prevented widespread use. A new signal synthesis method was needed that could better exploit the rapid advances in integrated circuit integration and computer architecture. Since sinusoidal summation models involve spectral descriptions, the key to an efficient new algorithm for additive synthesis is an efficient transformation from frequency to signal domain. Although the Fast Fourier Transform (FFT) was widely known and used since its rediscovery and introduction in 1965 (Cooley and Tukey, 1965), the challenges to its use for continuous synthesis of multiple sinusoids were not surmounted until the 1970’s. In a little known 1974 thesis, R.H. Davis (Davis, 1974) pioneered the two essential features of a synthesis window and overlap-add process. The first musical application of the weighted overlap-add inverse FFT method is described in a book by Chamberlin (Chamberlin, 1980). The benefits of the method are not obvious from this exposition because of the poor performance of the triangular and sine-squared windows suggested and a lack of affordable computers for the FFT calculations. The next important development came from the speech research community with the introduction of sinusoidal models for speech coding (McAulay and Quatieri, 1985). The inverse FFT method was applied to synthesize sinusoidally coded speech in 1988 (McAulay and Quatieri, 1988). In 1992 George and Smith described a musical tone synthesis scheme using the inverse FFT (George and Smith, 1992). By the early 1980’s the theory of transform domain synthesis of sinusoids and noise was well developed and had been applied in speech, music and other applications. More widespread application of this theory would require algorithms that efficiently exploited available computing machinery. In 1987 Rodet et al. developed tools for musical signal processing on an array coprocessor attached to a Sun workstation (Eckel, et al., 1987). Depalle and Rodet (Depalle and Rodet, 1990) developed an additive synthesizer based on the Inverse FFT for their musical workstation. This was the first real-time transform domain music synthesizer. By the early 1990’s workstations and desktop computers were fast enough for real-time implementations of additive synthesis with hundreds of partials (Freed, et al., 1993). Implementations of spectral line broadening in the transform domain require a frequency domain description of a modulated sinusoid. The analysis side of this problem was addressed by Marques and Almeida (Marques and Almeida, 1986, Marques and Almeida, 1989). Tabei and Ueda (Tabei and Ueda, 1988) explore the synthesis issues and Goodwin (Goodwin, 1997) sought efficient algorithms for non-stationary sinusoids (Goodwin and Kogon, 1995, Goodwin and Rodet, 1994). Unfortunately the key optimizations that make sinusoidal synthesis so efficient in the transform domain depend on the narrow band property of a constant frequency sine wave. This author has developed a novel compromise (Freed, 1997) for synchronous noise synthesis by adding random values to the phases of transform values for each bin in the transform associated with each sinusoid. Transform-Domain Additive Synthesis The computational kernel of transform domain sinusoidal synthesis is illustrated below: